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(54) PARALLAX IMAGE PICKUP DEVICE AND PARALLAX IMAGE PICKUP METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To obtain a parallax image 
pickup device that can detect a parallax amount with 
high accuracy so as to obtain accurate depth 
information of an object. 

SOLUTION: The parallax image pickup device is provided 
with an optical image forming section 12 with a single 
optical axis that forms an image of an object, a light 
receiving section 20 where a plurality of light receiving 
elements are placed and on which the image of the 
object is formed by an optical image forming section 12, 
a light passing section 14 with 1st and 2nd openings to 
emit the light passing through a 1 st and a 2nd area of 
the optical image forming section 12 to the light 
receiving section 20, an image pickup section 16 that 
allows the light receiving section 20 to simultaneously 
pick up a 1st image resulting from the object passing 
through the 1st opening and a 2nd image resulting form 
the object passing through the 2nd opening, and a 
distance calculation section 38 that calculates the 

distance from the optical image forming section 12 to at least one point on the object. Then the 
parallax image pickup device can simultaneously pick up a plurality of parallax images obtained 
when viewing the object from different points. 
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»gH«*JM&Lfc*£fc:o^-c, /*y3 6, ays? 

tttt}«|5 2 2 S geffi^!+^g|5 2 4M^iW-»g|5 2 6 

[0061] ^*P3 6I1 »^g|5 1 6 jJSfHfcLfc* tt 

ISMilHI;3 7(4, i$^#A, BMCorlfffeS. 
[0 0 6 2] v'ttajgB 2 2 It, * * V 3 6 ^iblilt 

[0 0 6 3] g 2 4 (4, 31 5/ vWf 23t 

frA, BMCW^SWt^m^TV^m^tl© 
#5£fi*£© g afSM©ffi£te» Lfc, »¥#£{*© g B 

[0 0 6 4] m£*tfeffl» 2 6 (4, 1T¥f*A, BMC 
© ft 3 tbT I * 5^M^»tSS»to^ 

[0065] ga*g|g©f(£-**5#y!, gafipwfit 

ftOEIt^ffflWSrJWi-fSftaicov^TH:, #l©£ 

[0066] -m^ m&zMznmmma-t, m^it 

flW©tff»(4, ;^XSBt45 0 *H16Jg«^J:ix 
(4, *9ffoft*W*©=y^**fifflbfc±"C» ^y^ 5 

§, IEIt«l^*li*o'^T, *^ft©IE«§#*£*tB 
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[0067] m-mm 3 ) 0 7 a, #*w©i8 3 ©n 

«^B|-efc5„ **»^ffi©;M 7(4, J6¥Wl2 
i, ftilffiSBl 4 t, iJy~7^(M—l 9 t, §7fc§|5 
20i, 7^vU^ — ,|g»)S|5 4 2 i, A/DIM3 4 
k , * * y 3 6 t , S3ff+j|$|5 3 8 i , ffeij«g|5 3 9 t, 

[0 0 6 8] 17 iCjoV^T, Jt^MSiMB 1 2 , ftffiffiffi 

1 4, #7 — 7^A-;?-l 9, S3tgl$2 0, A/D^ife 
10 §15 34, ^!)36, EMti-JIA 3 8 , XtflEft»4 0 

5„ ^xraftotmns 3 9 (47 a M-mm 

4 2 SrSI»i-5«^H:, % 1 ©HJS^li©fH»» 3 9 k 
IWCT?*>5*»e>, #a§|5^©E0^^t s B&1-5 o 
[0 0 6 9] y4fr?~-mW)M4 2iZ, 7fc»il§l514£ 
#fS£-ar3r t^T-#5„ fH»g|5 3 9 (4, 7^7WHK 

[0 0 7 0] 7^^^-.|g«)S|5 4 2(4, *r?ft©ffiMM 
fc«rSHfe-*-3»3\ 3tiiii«IJl 4 Sr^S^ 1 2tj>& 
20 gftStf 2 0(^MS3feS§rt fc#»£*5 0 m^4'9, * 

[0 0 7 1] 4fc7-f^-I«»4 2li, S^f*?r« 

2 0(c^55tSS^(-#B)^*5„ rtUiiD, */7(4 
[0 0 7 2] IIMiS^^«^fr©SgBI^*ft***tti 

[0 0 7 3] iiitf, ]R l ©mffi»«i (4 

30 itv\ R-©*^Srfflv^T, ^Hft©jftffM^ 
fr©«ft^f 5 rt^-C#5„ Ufc^oT, S^ff©Sg 
BI^*liJ^MS^#:©»ft('t5V^T, #ii©*^^ft 
§1512, *7-7-f;^-19MSM2 0^fflf 

[0074] mmmA) ins (4, «§^©»4©ji 

Mfim\U%Ztmm@iWi&%iW:<7>-fflk Lxn%/?<D 
*J*H-C*)5. *«»«©* ^7(4, £4«MMB12 
t, 7tiiii§l5 1 4 £, 77 7-7^-^^-1 9 £, S:7t§l5 
40 2 0 £, «f B iE»3§l5 4 4 £ , A/D^§)53 4 £, 

y 3 6 t, SgRfftl^ 3 8 t, $IJ«H53 9 t, fS«§|?4 
0^*-f-5„ 

[0 0 7 5] BI8^*3V-C, ^ft^HkUl 2, 77 7-7 
^/U^-19, S7fe§|32 0, A/D»§|53 4, 7*^0 

3 6, SgBff gffi 3 8 , MI2«§I5 4 0(COV^T(4, fR 
loHlt»«4:IBIC-e*>5ii»e>, RW*r*l»i-«. 
Wffi©SJ»§|3 3 9 (4?&§,!g»]£|5 4 4 ^UY^-f^X^ 
14, »l©£*JgJft©tfJ#PSB3 9 tPC-CfeS^fj, * 
5i§B^©^*«B&-t-5 0 

50 [0 0 7 6] ft&iifrci 4(4, mis i2MS3«l 
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P£|S1 5 a, 1 5 bRtfl 5 cSvfi-fa. fSl.Rl«B2 
p£B 1 5 a , 1 5 b (4, 1 2 (Dt\-m&& 

l-^tt bft, ^3<73MPg|51 5 cfl, *^fe^f|51 2 CO 

#»*r*trtWSfcRrt&*u JK3 0HPW1 6 d4, JR 
iXVSB2©Hnffli 5 a N l 5 b J;!J*#v*«K*r# 

[0 0 7 7]fl, l2Mf3«iPSlil5a 1 15 

[0 0 7 8] $£Bftfl|U 4 14, ftiliggfl HO^l, 
^2S^3»MPg|51 5 a, 1 5 b 1 5 c to%m 

[0 0 7 9] «£fP B I**jg|34 4 (4, S^frW^HH^tr* 
1 4 ©US 1 Xtf* 2 CDH Rg|5 1 5 
a, 1 5b*«ftftK:H$ % JffaoHpfVi 5 c&«ft 
wtvt4 9. #;*9f4tt¥ft©$itiliffe£ 

[0 0 8 0] ±£&iflBI!)gft4 4 (4, &^#£»&rf 5 
JtafflSSl 4W*3©MPaBl 5 cSrM^WfcM 
t, ^lXtf®2©MPai 5 a, 15bit*fi!lCB 

[0 0 8 1 ] «giBfca»bft^#<0E«#tftt«*rJ|Effl 
L, *9#0«K*#S:lllE-r8«Ha^oV^Ttt«10 

[0 0 8 2] ±fBt?f4, ftiliigB 14®|1 2 <D 

PJRS|315a, 1 5 b#, |§3tf>MR951 5 c <Dft-M\C 

H2©MRg|51 5 a, 1 5 bSr*LTt <tV\ 
[0 0 8 3] d©i§£\ £f£S& **«i-5^, fg3<7> 
fflpgpi 5 c**ftWfcB!t, mi XV* 2 »SP Si 

5 a, 1 5 b^mawtiiv^, K^flcsrjwfe-ra. tt 

¥#&*ffe1- £R#(4, SbK#3©ISR»l 5 c*H^ 

[0 0 8 4] r©4 5 l-ftiliSSSl 4 <D$& 1 RXffR 2 £> 
|?Pg|515a, 1 5 bSr^3 0lR$|51 5 cWF^ftlJtcl^ 
jg 3 ©M P 1 5 c Sr*^*gftgB 1 2 

[00 8 5] *mt»«l;:J:*Ltf, ftililSB 1 4 ^r*^ 

1 2 , JfcifiiaWi4, U7-7 4)V9 
-19, XlJtfi#»2 0«:«BU 
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* * 5 jStSis ft v % fc ft , * * 7 £ AMiti- 5 r t # 1? 

[0086] (mmwrn 5 ) g9it *3gej©s? 5 ©n 
mmimzz *7<Dm$Lmx°hz><, 

7(4, %3gB4fcJtfeffi lOt, ^X52h JtSS^fJ 
3 t, &9 5 4 i, y i/-vyX5 5i, ix-v 5/ 
^— 56t, *7-7^/^-58h CCD60h 
v/U^TV^f— 3 2 t, A/D^gP3 4t, ^ * V 
3 6 i, SBWH?g|53 8 t, fWs|53 9 t, fEftgM 0 

10 tSvPrtSo 

[0 0 8 7] H9(dt5VT, EI 2 i ID— ©?£-!§•£# L 
fc, uyX5 2, $9 5 4, 6, A/D'l 

tfeW 3 4 , y 3 6 , ESI!+»g|5 3 8 , fgffl& 3 9 & 
tJ«SSft$B4 0H:oV*Tf4, Siroilili^Dtfc?) 

[0 0 8 8] X-MMl&MtDjJ * 7(4, wyX5 2lrto 
X&^ft&fe&i-Zo Jtft^ffiH^ 5 3 (4, 1/^X5 2 
%iiilLfc7fe©7t?S^^#lt, $9 54, v-^y^-5 
6S.t) ? X)7 — 7 5 8^?TLTCCD6 0 (Cg:* 

20 y U--l^X5 5 Sr^LTfflUlJ^ftft^ 

1 OfcA^^^STtt^fJi-S, 

[0 0 8 9] ft^piff W i o (4, ft i ©Hi!»fi©> 

(4, !ll/-l/yX5 5 LT^iffllfe 

SHfeffi i o a* § n 5 o ti^s^aftai io <omi& t 

[0090] nmmm^^mM^mu &z#<om 
«t#*£*tBu **7tomtM*m\m^z>>)m.\im 
i (ommm t m c t- fo § a» & k w 5 = 

30 [0 0 9 1] y u-v>X5 5 fiftSrffi/J^i- Slf/hy u- 
-|ffcoTU<> SSeWiMHfcto 1 0 

t§CCD6 0f4Sf J f$:<O#V\ ISUftCCDffco 

yX5 2^T, CCD6 0t(4, ffttftOiKVMft? 
frwpjft^ffe^-iir, ftttBIMWfc* 1 0 T»f±, M&& 

[0 0 9 2] **»ffi©*^ ytiixff, vyX5 2 

5 u vXSr#a<k Lfc r t (c 4 5 , * ^ 9 Sr/hfflflrfS 

[0 0 9 3] (HJ6^ffi6) 01 0(4, «95©H6© 

S^tS^SB 10t, 2 t, gJt) 5 4 i , y 

^-56t, ttft§P46i:, A/Dliffi34t, 
y 3 6 t, Eg»f ^SP 3 8 t , SJ«3 9 t, fe^SP 4 
50 0^#1"S„ 
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[0094] mio (~ib'^T, m 2 k lea — ©??-§-&tt-L 

&g|53 4, ^^y36, EgKtf+li^3 8, 

tfE4ftSB4 ot'oi^Tii, US l ©itllt I?! CTfeS 

[0 0 9 5] Si lOlt * 1 <0%MM1&<0 

mmm&mmn ioxhz> 0 ummm.mmn 1 o 

« i-s saasiH i <»%mm t m zx*hz frbmw &k 10 

[0 0 9 6] ftfeftu 6 fi, K»#©««r*fl:*RiS:K 
[0 0 9 7] *H»fficoiS^*y 7(CJ;tbtt\ 

<5^T, ±S?»¥fcSr J: !>*ISKttffl^$ 

|K01AU«jfcl«m«oW^^iOlME«rai£^fT 20 
V\ **ft:SrliJlrt-5riaST?#5 0 

[0098] mMBMD mi in, *%m<Dm7(o 

WIIWftaHfcSlJ 1 0 k , wyX52i, K954h * 
7-7-f/^-5 8i, CCD60i, A/D^gB3 
4 a 34bL ^*|)36a XU 5 36bi, SgHtft 
*g|5 3 8 t , Mfffi 3 9 i , GftgB 4 0 5. 

[0 0 9 9] 011 (c&i^T, S2 irW— ©?f-5§-S:ttL 

ft, 1/^X5 2, S'9 5 4, *7-7^;^-5 8, C 30 
CD 6 0, A/D'^g|53 4a, 34b, ^ y 3 6 
a , 3 6b, Eg»f+#a5 3 8 , fflfflffi 3 9 XtWEfikW 4 

&&»<oWi®<?>m»*mamiz.ft5 hi 
[oioo] %snttaHKs i o t±, hi (ommmm<D 

■CtHfc*Jlfc**HifcH:, A/DaE*S8 3 4 aWct 40 
fy?Mt^W@kZix, ^!)36a (cfS'lt^ixSo 
gg«W-#SP3 8(i^*y 3 6 a a>g>tt£Bfe*K*ti] 

[0 10 1] mSWft/5^ffi¥f*<OSEgf^*Sr»ffiL, 

[0 10 2] uyX5 2#lSlki-5*9*OPHfcl4CC 
D6 0(cS7t^ti, A/D'£&ffl5 3 4 b (Ciotfv 5 ^ 

/Wt#K3fc*3*u 3 6 bkifcaftstis. 

ftfcffi^#:OH{ti±fB®g|5 4 0(C^«^ft5„ £S8M 50 
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0 f± tff^X— 7\ MOStfD VDf©t2ilft:ffe-o 
[0 10 3] *^*3glBWK'7 f ^-*^7tJ:tl«:» 

[0104] (^HJ|?ffi8) 01 2f±, *»W(DH8© 

[0 10 5] *XllBgtt0>rt&tt§gftMU f*jffi^7 0 
t, ff^a§|57 2 i, fea$|54 0 * = * 7 4§r# 

k , CCD 6 0 3t^*l|felfB 1 2 i , JtifiiSS 1 4 

[0 10 6] faWtL7 OWffrtStt, fe^^-7'/U7 8 
4r#L, CCD 6 0X0*1*^1 6 roffl^*^:ff^5re 

arts. nmmmzwiiMLi-ztzibnft^mui 2t, 

1 4 t , l 6 ©«j$f±, H l <r)%mm 

©«3iBfc*iM& i o tffcs. 

[ 0 1 0 7 ] fa^aSB 7 2 {4, CCD 6 0XU««^gP 

HJU **#©B*fcPMM!Ha«r*U ^?7 4t;iB 
7JL, 0(ctti7lt-5 0 fl|-W!Ha*B7 2f±*=^ 

■rs^asH i nmmmkm tT»*>*a»e>Rw«r*i» 

[0108] -e-^ 7 4 l«¥f#cD^£i&¥te©S»tif 
«t tt{-*^-f-5o liLlt*-* 7 4 («¥#:©iz:#:B 

[0109] *HJt^^T-fl, C C D 6 0 Xl/a^ft: 1 

6 *ftnm7 o (D^m^miftztK mmmmaztn' 

ffi)t$tbJ5CV\ CCD60SOlMl6?rrtSi70 

X^gglt, ^X5 2SU 5 7tWf^l 2^«LfcB 
1» b—U-VXtdtfli^*, F fe 3ffi^7 0W^f|5T-C 
C D 6 0 Xl>*a^* 1 6 (iltft £ *T t i V \, 

[oiio] ^i^fi^ffit.tsrtftftKlttiixtfx 
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[0111] (nmmm 9 ) amt *3pj3<7?ss 9 » 

So 

[0112] 111 l 3 \z.artz > yv— 5^ — 1-£#h6 s l& 

35 1 6#, ^^5,^^Wffl«(ct5(tS7t<0®ig^$iJPS 

(SI 0 2) , 3iy5>ttW«|J2 2iJS, g3t35fC$g&£;jX 10 

1~5 (S 1 0 4) „ 
[0 113] gE4lHt|-JFffi2 4tf, W3e©«««3:y 
vUHftO (x, y) JfcKSfcJottSM* (i, j) K*S 
rtSHKSx-^WttfcF (i, j) iU iy5?W(fcO 

(s i o 6) „ tigs 77 do(', sus^m 7j *f± 

FJT£»<tMAX 1 tttl^lKtomctZ, *@M{i£<i>l4 0 
Wt«*M$m i n{±Bff^filMAX2lC^tl 

^hWMik-fZo z rt-MAX i (i^ffi^tt-sm^iro 20 

liH, MAX2l*|*ffl£;h,5ffiBBfirofcJ:9 * 
[0114] I BfHBBW-JISB 2 4 !4, ftgft » * 1 

•fy^y^yM (sios) , ffiMf($^^S(-<to 
-ctmi-5 (s 1 1 o) . 

0=E i 2 j [F (i , j ) — F ( i — 7j , j ) ] * 2 
[0 115] tt£ft4fttttffi2 6j5^ 

oft/Nit Om i n j; <9 fc/h^v^fj, ftBHi<£>ft/JMi$ 

m i njqS£©*BKtt*«:ftAU SH*©*aHrj * 30 

\z%.&<z>UmMv&tt*.irZ> (S 1 1 2) . aftwfflU 
*ijiiSEr5fe*J:!>*f <*v^»£\ si08!;l?), s 

10 8, S 1 1 0RXIS 1 1 2<Dtim&$&*)Mt<, 

[0116] nmm^tan 2 6 it, m&^mmntm 

116), ti^lw^ififiT, *fi^ , t> %^^M 
5 (S 1 1 8) . flHt*0>*fc*<5v*T, $?¥ft:»8EBt 

w&tgm-tZo 40 

[oii7] #n*l?ffi©&nwft*te;frftfc±*itf, 

[0118] mmmmi 0) mi 4«u #3s?ioii 
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[01193014 fcfctfS^n— ^ — b£r#J$Lfr 

351 6^, S*S^£Jl^-©«*jcfcj3»t53t<7)ifiia4rfi|IB 
-T3)feiIifi35£^LT, *^ft©W«S:S3fe*B5t;:««i-t 
5 (S2 0 2) „ ay5?iftttHBJ2 2jJS, S^^JiftS 
*LfcB«McJ3ft<5«3e©tWiE*WlL (S 2 0 4) , S 
f^^Wftlilco^T, 3iy^B**:aHii-5 (S 2 0 
6) . 

[0 12 0] gatiHI+^35 2 4 ^^ 

ftEttfcfcS-frsfctf «g*tfettJgB2 6^, 

ysmft*#«Sfci:«>aae*«:J|ttiJL (S 2 0 8) , 
«*ii*:*9#*-eorai*r»HI-*-5 (S 2 1 0) 0 

[0121] taM(-a^v^Tti^*^^ttii-5^as 2 

0 8lt 111 3£33tt5S 106MS1 1 

[0 12 2] |i*S^tB35 2 6(4, §*35(e:|t|g!$ftfc 

(S212) , -r^xn&wmmz-^xym* 

iliRU S 2 0 6 S 2 1 0 * 9 St. 
[0 12 3] -t-iTWi&S&ffl^t-oVNTSgBISrH-^L 

3 9 *£?#©ilite©££*» 
S^T, ±K»9#^«iW-f5 (S 2 1 4) . fiij#33 

(S 2 1 6) „ 

[0124] MfflgB 3 9 (4, 7 w frPmWlU 4 2 ^f&JW 
S&^#S)£* (S 2 1 8) , »^i*£jgirf 3 (S 2 

20) , -ttz-mnrnMnrnt tt, 11*353 9 (4, S2 

1 8(CioV^T, ^bbP«i354 4^flLT« ^;iiS35© 
[0 12 5] *HjMUKffi©S^W«»fll*iSfe^J:ittf, 

[0 12 6] £t±, *»W«riHfiO»«t:fflV^T»WL 
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2 0 0 1-16 6 12 



19 



20 



[0 12 7] 

fcflHfcU M£ft*M*>&&£*«rtttt]U 10 
[BiS®ffi¥ttR9n 

[0 1 1 «*o|i^Wt*{f £E«Mi 
[02] *«^©JISi ^ 7 ©«^H 

[El 3] tt£aftftft«*ft>ttj£H 

[04] umm&.<nttimM<DM!8.m 

[05] gB*ai»»tmiiJiroimi*«fe»»wig 

[0 6] **W©fS2©JOS»*©*^9lC*J»t* : S 
MWf*O$O ! SgPWlt)^0 
[0 7] *^KW^3WHJt^liro*>7<D#|^0 20 
[0 8] «iOf 4©»1©*^ 7»M 
[0 9] *^SH^(75^5t73^»ffi©^^7(C«J$[g] 

[0io] *%®<Dfis 6 vmtmmvmBi* * 7 <om 

J&0 * 



* [011] JfWftoW, 7 w*Jfe^H»ro fx** ^7(0 

[0i2] *%mv>m s omt^tiortttimito* 
j&0 

[013] *3§BJ!oJ» 9 0mgtto?IU£PMtiMt£ 

[014] #3gg§©$& i o o%Mitm<Dmmm®M\ik 

1 o mmmmm^ 1 2 u 
1 4 Ttiiii^p 15a, b mu% 

1 6 ftfeft 

19 TJy — 2 0 gftgC 

2 2 iy^ttaSS 2 4 SBftBStf 

2 6 KH*tfeW«f» 3 2 -riVf-yv 

3 4 A/D^&SJ 3 6 

3 8 EKfJ-gffi 3 9 MfpfflS 
4 0 BEfttft 

4 2 y-i^V-WMU 4 4 tt&KSigis 

4 6 mtU 5 2 UVX 

5 4 5 6 *s*y9 — 

5 8 t!7~- 7 41V?— 6 0 CCD 



[01] [07] 
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-40^ 



[09] 




(14) 



#f!fl2 0 0 1 - 1 6 6 1 2 



I 



15a 



I— 15c 



5b 



[08] 



[011] 
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up 

16 



9h 



i-44 



ft 
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J 54 

is y ccd 
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[Ell 2] 



[mi 4] 



m&v *'*_J— S202 

J^S204 




12 14 16 7 ' 8 



I jggjft^ttffl | — S214 



S216 



7<»*g3fcMH1-lsfH» I — S218 



flE^ffg>fl5& [ — S220 
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[HI 3] 



ftgWftcftlsiri — S102 



lHSi<Pftffi | — S104 



fci.j) 

gMu^O, Tj *=MAX1 
ft HIS *=0, Omin=MAX2 








— S108 






<t> = r z (F(i. j) - 


F(i-77. j)) 2 



— S106 



S110 



if Q<<frmin then Omin=<fr, tj * = f? 




S112 



S114 



%ia>) »(&niA f ~~ sii6 



mmoftn — si 1 e 



— 2F065 AA06 DD02 FF04 FF09 JJ03 

JJ26 LLOO LL04 LL10 LL21 
LL30 NN11 PP21 QQOO QQ03 
QQ13 QQ23 QQ24 QQ27 QQ28 
QQ41 

2H059 AA09 AA18 AA24 AA35 
5B057 BA02 BA11 BA15 BA21 BA29 
DA07 DB03 DB06 DC16 DC34 
5C061 AB03 AB12 



